{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# CSV Analysis Example"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a demo notebook showing how to use the `orion.analysis.analyze` function on a CSV signal."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Load the data\n",
    "\n",
    "In the first step, we setup the environment and load the CSV that we want to process.\n",
    "\n",
    "To do so, we need to import the `orion.data.load_signal` function and call it passing\n",
    "the path to the CSV file.\n",
    "\n",
    "In this case, we will be loading the `S-1.csv` file from inside the `data` folder."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import logging;\n",
    "\n",
    "logging.basicConfig(level=logging.ERROR)\n",
    "logging.getLogger().setLevel(level=logging.ERROR)\n",
    "\n",
    "import warnings\n",
    "warnings.simplefilter(\"ignore\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>timestamp</th>\n",
       "      <th>value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1222819200</td>\n",
       "      <td>-0.366359</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1222840800</td>\n",
       "      <td>-0.394108</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1222862400</td>\n",
       "      <td>0.403625</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1222884000</td>\n",
       "      <td>-0.362759</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1222905600</td>\n",
       "      <td>-0.370746</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    timestamp     value\n",
       "0  1222819200 -0.366359\n",
       "1  1222840800 -0.394108\n",
       "2  1222862400  0.403625\n",
       "3  1222884000 -0.362759\n",
       "4  1222905600 -0.370746"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from orion.data import load_signal\n",
    "\n",
    "signal_path = 'S-1'\n",
    "\n",
    "data = load_signal(signal_path)\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Detect anomalies using a pipeline\n",
    "\n",
    "Once we have the data, let us try to use the LSTM pipeline to analyze it and search for anomalies.\n",
    "\n",
    "In order to do so, we will have import the `orion.analysis.analyze` function and pass it\n",
    "the loaded data and the path to the pipeline JSON that we want to use.\n",
    "\n",
    "In this case, we will be using the `lstm_dynamic_threshold.json` pipeline from inside the `orion` folder.\n",
    "\n",
    "The output will be a ``pandas.DataFrame`` containing a table with the detected anomalies."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 7919 samples, validate on 1980 samples\n",
      "Epoch 1/5\n",
      "7919/7919 [==============================] - 35s 4ms/step - loss: 0.1959 - mse: 0.1959 - val_loss: 0.3439 - val_mse: 0.3439\n",
      "Epoch 2/5\n",
      "7919/7919 [==============================] - 34s 4ms/step - loss: 0.1915 - mse: 0.1915 - val_loss: 0.3147 - val_mse: 0.3147\n",
      "Epoch 3/5\n",
      "7919/7919 [==============================] - 34s 4ms/step - loss: 0.1909 - mse: 0.1909 - val_loss: 0.3534 - val_mse: 0.3534\n",
      "Epoch 4/5\n",
      "7919/7919 [==============================] - 34s 4ms/step - loss: 0.1873 - mse: 0.1873 - val_loss: 0.2893 - val_mse: 0.2893\n",
      "Epoch 5/5\n",
      "7919/7919 [==============================] - 34s 4ms/step - loss: 0.1891 - mse: 0.1891 - val_loss: 0.3037 - val_mse: 0.3037\n",
      "9899/9899 [==============================] - 13s 1ms/step\n",
      "9899/9899 [==============================] - 12s 1ms/step\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>start</th>\n",
       "      <th>end</th>\n",
       "      <th>score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1228219200</td>\n",
       "      <td>1229472000</td>\n",
       "      <td>0.656399</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1400587200</td>\n",
       "      <td>1404172800</td>\n",
       "      <td>0.166991</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        start         end     score\n",
       "0  1228219200  1229472000  0.656399\n",
       "1  1400587200  1404172800  0.166991"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from orion.analysis import analyze\n",
    "\n",
    "hyperparameters = {\n",
    "    'keras.Sequential.LSTMTimeSeriesRegressor#1': {\n",
    "        'epochs': 5,\n",
    "        'verbose': True\n",
    "    }\n",
    "}\n",
    "\n",
    "pipeline = 'lstm_dynamic_threshold'\n",
    "\n",
    "anomalies = analyze(pipeline, data, hyperparams=hyperparameters)\n",
    "anomalies"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
